1. Field of the Invention
This invention relates to an image processing method and apparatus. This invention particularly relates to an image processing method and apparatus, wherein sharpness of an image is enhanced, while graininess (high frequency noise) of the image is being suppressed. This invention also relates to an image processing method and apparatus, wherein graininess of an image is suppressed.
2. Description of the Related Art
With techniques for reading out an image (or a color image) and reproducing the image as a visible image, an image having been recorded on photographic film, a print, or the like, is photoelectrically read out with a sensor, such as a charge coupled device (CCD) image sensor, and an image signal is thereby obtained. Various image processing operations are carried out on the image signal, and a visible image is reproduced on a print, a cathode ray tube (CRT) display device, or the like, from the image signal, which has been obtained from the image processing. Examples of such image processing include processing in frequency domain, which acts upon predetermined frequency components contained in an image, and gradation processing, which acts upon image density, or the like.
Examples of the frequency processing in frequency domain include sharpness enhancement processing for suppressing blurr of a contour in an image, and graininess suppression processing, such as smoothing processing, for suppressing noise (graininess) due to grains of a photosensitive material, or the like.
As the sharpness enhancement processing, there have heretofore been known unsharp masking processing, high-band enhancement filter processing, processing disclosed in, for example, U.S. Pat. No. 5,739,922, wherein high frequency components of an image are enhanced and middle frequency components of the image are suppressed, and the like. As the graininess suppression processing, there have heretofore been known median filter processing, hysteresis smoothing processing, noise removal processing with iteration, graininess suppression processing utilizing a morphology operation, and the like.
With the sharpness enhancement processing, the sharpness of an image can be enhanced, but the problems occur in that the graininess of the image is also enhanced and a rough feeling of the image remains. With the graininess suppression processing, the graininess of an image can be suppressed and a rough feeling of the image can be reduced, but the problems occur in that the extent of the sharpness of the image becomes low.
Therefore, a need exists for an image processing method, with which the graininess can be suppressed and, at the same time, the sharpness can be enhanced. There have heretofore been proposed various image processing methods, which aim at simultaneously achieving graininess suppression and sharpness enhancement. (Such image processing methods have been proposed in, for example, U.S. Pat. No. 4,812,903, and Japanese Unexamined Patent Publication Nos. 63(1988)-26783 and U.S. Pat. No. 5,739,922.) However, with the proposed image processing methods, actually, both the graininess suppression and the sharpness enhancement cannot simultaneously be achieved efficiently.
For example, the technique disclosed in U.S. Pat. No. 5,739,922 aims at suppressing the graininess and enhancing the sharpness by carrying out processing for enhancing high frequency components, which affect the image sharpness, and suppressing middle frequency components, which affect the graininess that gives a rough feeling. The disclosed technique utilizes the characteristics in that the degree of correlation between colors is low at a grainy area and is high at a not-grainy area (edge area). Also, with the disclosed processing, an image is spread in the frequency domain, intensities of middle and high frequency components are controlled, and the extent of them is kept low with respect to an area having a low level of color correlation. In this manner, graininess suppression is carried out. Specifically, with the disclosed processing method, since the factor of the connection between adjacent pixels in an image is not taken into consideration, the extent of the intensities of frequency components changes sharply at a certain area in the image, and it cannot be felt that the graininess has been suppressed. Therefore, the effects of graininess suppression are small in comparison with the effects of the sharpness enhancement. From the aspect of the effects, the disclosed processing is actually the sharpness enhancement processing, instead of being the graininess suppression and sharpness enhancement processing.
The primary object of the present invention is to provide an image processing method, wherein graininess suppression and sharpness enhancement for an image are capable of being simultaneously achieved efficiently.
Another object of the present invention is to provide an image processing method, wherein graininess suppression and sharpness enhancement for an image are capable of being simultaneously achieved efficiently, and graininess suppression processing is not carried out with respect to a predetermined image area that is not to be suppressed.
A further object of the present invention is to provide a graininess suppression image processing method, wherein positive graininess and negative graininess are suppressed accurately.
The specific object of the present invention is to provide an apparatus for carrying out the image processing method.
A first image processing method in accordance with the present invention is characterized by carrying out sharpness enhancement image processing and graininess suppression image processing, respectively, on an original image signal, carrying out processing on image signal components of two image signals having been obtained from the sharpness enhancement image processing and the graininess suppression image processing, which image signal components represent corresponding pixels in two images represented by the two image signals, and thereby obtaining a processed image signal.
Specifically, the present invention provides a first image processing method, wherein predetermined processing is carried out on an original image signal (or each of original image signals representing an original color image) fin, which is made up of a series of image signal components defining values of pixels constituting an original image, and a processed image signal fout is thereby obtained, the method comprising the steps of:
i) carrying out graininess suppression image processing on the original image signal fin, a graininess suppression processing image signal f1 being obtained from the graininess suppression image processing,
ii) carrying out sharpness enhancement image processing on the original image signal fin, a sharpness enhancement processing image signal f2 being obtained from the sharpness enhancement image processing, and
iii) carrying out processing for obtaining the processed image signal fout in accordance with the graininess suppression processing image signal f1 and the sharpness enhancement processing image signal f2, the processed image signal fout being obtained with operation processing carried out on image signal components of the graininess suppression processing image signal f1 and the sharpness enhancement processing image signal f2, which image signal components represent corresponding pixels in two images represented by the graininess suppression processing image signal f1 and the sharpness enhancement processing image signal f2.
Details of the operation processing may vary for the pixels or may be common to all of the pixels. Also, the details of the operation processing may be selected from a plurality of kinds of predetermined operations and in accordance with, for example, relationship with the value of the original image signal fin, the kind of the image, or the position of a picture element. Alternatively, information representing the details of the operation processing may be inputted from the exterior each time the processing is to be carried out.
Specifically, as the operation processing described above, the operation processing defined with Formula (1) shown below may be employed:
fout=w1xc2x7f1+w2xc2x7f2xe2x80x83xe2x80x83(1)
wherein w1 and w2 represent weight factors, w1+w2=1 (0 less than w1 less than 1, 0 less than w2 less than 1), fout represents the processed image signal, f1 represents the graininess suppression processing image signal, and f2 represents the sharpness enhancement processing image signal.
Specifically, with the operation processing defined with Formula (1), the processed image signal fout is obtained by weighting the graininess suppression processing image signal f1 and the sharpness enhancement processing image signal f2 and adding the weighted image signals to each other. In the operation processing defined with Formula (1) shown above, the values of the weight factors w1 and w2 may be altered in accordance with the relationship with the value of the original image signal, or the like. Alternatively, the weight factors common to all of the pixels may be set.
Each of the weight factors w1 and w2 should preferably take a value falling within the range of 0.2 to 0.8, should more preferably take a value falling within the range of 0.3 to 0.7, and should most preferably take a value falling within the range of 0.4 to 0.6.
Alternatively, as the operation processing described above, the operation processing defined with Formula (2) shown below may be employed:
fout={|f1xe2x88x92fin|(f1xe2x88x92fin)+|f2xe2x88x92fin|(f2xe2x88x92fin)}/{|f1xe2x88x92fin|+|f2xe2x88x92fin|}+finxe2x80x83xe2x80x83(2)
wherein fin represents the original image signal, fout represents the processed image signal, f1 represents the graininess suppression processing image signal, and f2 represents the sharpness enhancement processing image signal.
Specifically, with the operation processing defined with Formula (2), as for a picture element, at which the value of the graininess suppression processing image signal is equal to the value of the original image signal, the value of the sharpness enhancement processing image signal is taken as the value of the processed image signal. As for a picture element, at which the value of the sharpness enhancement processing image signal is equal to the value of the original image signal, the value of the graininess suppression processing image signal is taken as the value of the processed image signal. As for a picture element, at which the value of the graininess suppression processing image signal and the value of the sharpness enhancement processing image signal are not equal to the value of the original image signal, the addition through weighting with the difference between the graininess suppression processing image signal or the sharpness enhancement processing image signal, whichever has a larger difference with respect to the original image signal, and the original image signal is carried out, and the value obtained from the weighting addition is taken as the value of the processed image signal. The operation processing defined with Formula (2) is one of the examples, wherein the processing is selected for each of the pixels in accordance with the relationship with the value of the original image signal (fin).
As the graininess suppression image processing (and the graininess suppression image processing carried out by a graininess suppression processing means, which will be described later), it is possible to employ median filter processing, hysteresis smoothing processing, noise removal processing with iteration, graininess suppression processing (smoothing processing) utilizing a morphology operation, or the like. As the sharpness enhancement image processing, it is possible to employ unsharp masking processing, high-band enhancement filter processing, processing disclosed in, for example, U.S. Pat. No. 5,739,922, wherein an image signal is separated into low frequency components, middle frequency components, and high frequency components, wherein enhancement and suppression processing is carried out such that the high frequency components may be enhanced and the middle frequency components may be suppressed, and wherein the high frequency components and the middle frequency components, which have been obtained from the enhancement and suppression processing, and the low frequency components are combined with one another, or the like. As the graininess suppression image processing and the sharpness enhancement image processing, there should preferably be employed a combination of the graininess suppression processing (smoothing processing), which utilizes a morphology operation, and the sharpness enhancement processing (disclosed in, for example, U.S. Pat. No. 5,739,922, wherein the enhancement and suppression processing is carried out such that the high frequency components of an image may be enhanced and the middle frequency components of the image may be suppressed, and wherein the high frequency components and the middle frequency components, which have been obtained from the enhancement and suppression processing, and the low frequency components are combined with one another.
In general, the morphology operation processing is expanded as the theory of sets in an N-dimensional space. Also, the morphology operation processing is often applied to an image in a two-dimensional space (as described in, for example, U.S. Ser. No. 08/623,223, U.S. Ser. No. 08/818,039 and Japanese Unexamined Patent Publication No. 9(1997)-91421). The morphology operation processing will hereinbelow be described briefly by taking a gray level image as an example.
The gray level image is considered as a space, in which a point having coordinates (x, y) has a height corresponding to an image density value f(x, y). Also, a one-dimensional function f(x) corresponding to the cross-section of the gray level image is considered. It is assumed that a structure element g used in the morphology operation processing is a symmetric function of Formula (5) shown below, which is symmetric with respect to the origin.
gs(x)=g(xe2x88x92x)xe2x80x83xe2x80x83(5)
It is also assumed that the value is 0 in a domain of definition G, which is represented by Formula (6) shown below.
G={xe2x88x92m, xe2x88x92m+1, . . . , xe2x88x921, 0, 1, . . . , mxe2x88x921, m}xe2x80x83xe2x80x83(6)
In such cases, the fundamental forms of the morphology operation are very simple operations carried out with Formulas (7), (8), (9), and (10) shown below.
dilation; [f⊕Gs](i)=max {f(ixe2x88x92m), . . . , f(i), . . . , f(i+m)}xe2x80x83xe2x80x83(7)
xe2x80x83erosion; [fxe2x8ax96Gs](i)=min {f(ixe2x88x92m), . . . , f(i), . . . , f(i+m)}xe2x80x83xe2x80x83(8)
opening; fg=(fxe2x8ax96gs)⊕gxe2x80x83xe2x80x83(9)
closing; fg=(f⊕gs)xe2x8ax96gxe2x80x83xe2x80x83(10)
Specifically, as illustrated in FIG. 4A, the dilation processing is the processing for retrieving the maximum value in the region of a width of xc2x1m (which width is the value determined in accordance with the structure element B and corresponds to the mask size shown in FIG. 4A) having its center at a picture element of interest. As illustrated in FIG. 4B, the erosion processing is the processing for retrieving the minimum value in the region of the width of xc2x1m having its center at the picture element of interest. The opening processing is equivalent to the processing in which the dilation processing is carried out after the erosion processing, i.e., the processing in which the maximum value is searched after the searching of the minimum value. Also, the closing processing is equivalent to the processing in which the erosion processing is carried out after the dilation processing, i.e., the processing in which the minimum value is searched after the searching of the maximum value.
More specifically, as illustrated in FIG. 4C, the opening processing is equivalent to the processing for smoothing the image density curve f(x) from the low density side, and removing a convex image density fluctuating area (i.e., the area at which the density is higher than that of the surrounding areas), which fluctuates in a region spatially narrower than the mask size of 2 m.
Also, as illustrated in FIG. 4D, the closing processing is equivalent to the processing for smoothing the image density curve f(x) from the high density side, and removing a concave image density fluctuating area (i.e., the area at which the density is lower than that of the surrounding areas), which fluctuates in the region spatially narrower than the mask size of 2 m.
In cases where the image signal representing the image density value f(x) is a high image density-high signal level type of image signal, in which a high image density is represented by a high image signal level, the relationship between the image density value f(x) and the image signal value becomes reverse to the relationship between the image density value f(x) and the image signal value in the high luminance-high image signal level type of image signal. Therefore, the dilation processing, which is carried out on the high image density-high signal level type of image signal, coincides with the erosion processing, which is carried out on the high luminance-high signal level type of image signal as shown in FIG. 4B. The erosion processing, which is carried out on the high image density-high signal level type of image signal, coincides with the dilation processing, which is carried out on the high luminance-high signal level type of image signal as shown in FIG. 4A. The opening processing, which is carried out on the high image density-high signal level type of image signal, coincides with the closing processing, which is carried out on the high luminance-high signal level type of image signal as shown in FIG. 4D. Also, the closing processing, which is carried out on the high image density-high signal level type of image signal, coincides with the opening processing, which is carried out on the high luminance-high signal level type of image signal as shown in FIG. 4C.
In the manner described above, the opening processing or the closing processing with the morphology operation processing may be carried out on the image signal representing an original image. As a result, graininess (i.e., noise in the image signal) in the image can be suppressed (or removed). (Such features are described in, for example, xe2x80x9cMorphologyxe2x80x9d by Kobatake, published by Corona Co.).
The morphology operation processing, which is preferable in the image processing method in accordance with the present invention, will be described later.
The sharpness enhancement processing (disclosed in, for example, U.S. Pat. No. 5,739,922, wherein the enhancement and suppression processing is carried out such that the high frequency components of an image may be enhanced and the middle frequency components of the image may be suppressed, will hereinbelow be described briefly.
With the disclosed sharpness enhancement processing, an image signal is separated into low frequency components, middle frequency components, and high frequency components, and enhancement and suppression processing is carried out such that the high frequency components may be enhanced and the middle frequency components may be suppressed. The high frequency components and the middle frequency components, which have been obtained from the enhancement and suppression processing, and the low frequency components are then combined with one another, and a processed image signal is thereby obtained.
FIG. 5 is a graph showing how low, middle, and high frequency components are distributed. The term xe2x80x9clow frequency components, middle frequency components, and high frequency components of an image signalxe2x80x9d as used herein means the frequency components, which are distributed in the patterns shown in FIG. 5. The middle frequency components are distributed such that they may have a peak in the vicinity of ⅓ of the Nyquist frequency of the output in the reproduction of a visible image from the signal having been processed. The low frequency components are distributed such that they may have a peak at a frequency, at which the Nyquist frequency of the output becomes equal to 0. The high frequency components are distributed such that they may have a peak at the Nyquist frequency of the output. At each frequency, the sum of the low, middle, and high frequency components becomes equal to 1.
In the disclosed sharpness enhancement processing, the processing should preferably be constituted such that, after the separation, luminance components may be extracted from the high frequency components and the middle frequency components, and the enhancement and suppression processing and the combining may be carried out in accordance with only the luminance components.
Also, in the disclosed sharpness enhancement processing, the processing should preferably be constituted such that a specific color region may be extracted from the given image, and the enhancement and suppression processing may be carried out such that the middle frequency components, which correspond to the specific color region, may be suppressed even further.
Further, in the disclosed sharpness enhancement processing, the processing should preferably be constituted such that, after the separation, a calculation may be made to find a rating value of the middle frequency components and/or the high frequency components, and the enhancement and suppression processing may be carried out such that the middle frequency components corresponding to pixels, which are associated with the rating values smaller than a predetermined threshold value, may be more suppressed than the middle frequency components corresponding to pixels, which are associated with the rating values larger than the predetermined threshold value, are. In such cases, the high frequency components corresponding to the pixels, which are associated with the rating values smaller than the predetermined threshold value, should preferably be less enhanced than the high frequency components corresponding to the pixels, which are associated with the rating values larger than the predetermined threshold value, are.
The term xe2x80x9crating valuexe2x80x9d as used herein means a value, such as a correlation value between at least a single set of two colors, which are among red, green, and blue three colors, as will be described later, or a local area limited variance value of the frequency components of the image signal.
The rating value should preferably be a correlation value between at least a single set of two colors, which are among red, green, and blue three colors, the correlation value being calculated from the middle frequency components and/or the high frequency components, which are of the respective colors and correspond to the same picture element.
Also, the rating value may be subjected to a filter processing with a median filter, and thereafter the enhancement and suppression processing may be carried out by using the predetermined threshold value and in accordance with the rating value, which has been obtained from the filter processing with the medial filter.
Further, the calculation of the middle frequency components and/or the high frequency components and the calculation of the rating value may be carried out respectively in accordance with different colors, which are among red, green, and blue three colors.
Furthermore, the degrees of the enhancement and the suppression in the enhancement and suppression processing should preferably be determined by being selected from a plurality of enhancement and suppression processing conditions, which have been determined previously in accordance with image reproducing conditions in the reproduction of a visible image from the processed image signal.
The term xe2x80x9cimage reproducing conditionsxe2x80x9d as used herein means the conditions, which affect the characteristics of the reproduced visible image, such as the kind of the recording medium on which the given image was recorded (e.g., negative film or reversal film), the size of the print on which the visible image is reproduced, or a correction amount which is specified by an operator from a keyboard such that a desired image processing may be carried out.
As described above, with the graininess suppression in the enhancement and suppression processing disclosed in U.S. Pat. No. 5,739,922, since the factor of the connection between adjacent pixels in an image is not taken into consideration, it cannot be felt that the graininess has been suppressed. However, the graininess suppression processing utilizing the morphology operation is the processing on the image plane utilizing the isolation characteristics of graininess (graininess has low continuity of image signal (image density) with adjacent pixels). Therefore, with the graininess suppression processing utilizing the morphology operation, the connection between adjacent pixels can be kept good, the extent of graininess suppression can be felt visually, and large effects of graininess suppression can be obtained.
The present invention also provides a first image processing apparatus for carrying out the first image processing method in accordance with the present invention. Specifically, the present invention also provides a first image processing apparatus, wherein predetermined processing is carried out on an original image signal, which is made up of a series of image signal components defining values of pixels constituting an original image, and a processed image signal is thereby obtained, the apparatus comprising:
i) a graininess suppression processing means for carrying out graininess suppression image processing on the original image signal, a graininess suppression processing image signal being obtained from the graininess suppression image processing,
ii) a sharpness enhancement processing means for carrying out sharpness enhancement image processing on the original image signal, a sharpness enhancement processing image signal being obtained from the sharpness enhancement image processing, and
iii) a signal processing means for carrying out processing for obtaining the processed image signal in accordance with the graininess suppression processing image signal, which has been obtained from the graininess suppression processing means, and the sharpness enhancement processing image signal, which has been obtained from the sharpness enhancement processing means, the processed image signal being obtained with operation processing carried out on image signal components of the graininess suppression processing image signal and the sharpness enhancement processing image signal, which image signal components represent corresponding pixels in two images represented by the graininess suppression processing image signal and the sharpness enhancement processing image signal.
The operation processing, which is carried out by the signal processing means, may vary for the pixels or may be common to all of the pixels. However, the operation processing should preferably be set so as to vary for the pixels. In such cases, graininess can be suppressed appropriately and sharpness can be enhanced appropriately in accordance with details of a structure embedded in the image, or the like.
Also, as the operation processing, which is carried out by the signal processing means, the operation processing described above, which is defined with Formula (1) or Formula (2) shown above, should preferably be employed.
The first image processing method in accordance with the present invention should preferably be modified such that, after the graininess suppression image processing and the sharpness enhancement image processing have respectively been carried out on the original image signal, threshold value processing with a difference between the original image signal and the graininess suppression processing image signal may be carried out, and details of subsequent processing may be altered in accordance with the results of the threshold value processing. In cases where the degree of the graininess suppression is high (i.e., in cases where the difference is larger than a threshold value), graininess suppression processing may not be carried out. In cases where the degree of the graininess suppression is low (i.e., in cases where the difference is smaller than the threshold value), the operation processing described above may be carried out. (Such a modification constitutes a second image processing method in accordance with the present invention.)
Specifically, the present invention further provides a second image processing method, wherein predetermined processing is carried out on an original image signal (or each of original image signals representing an original color image) fin, which is made up of a series of image signal components defining values of pixels constituting an original image, and a processed image signal fout is thereby obtained, the method comprising the steps of:
i) carrying out graininess suppression image processing on the original image signal fin, a graininess suppression processing image signal f1 being obtained from the graininess suppression image processing,
ii) carrying out sharpness enhancement image processing on the original image signal fin, a sharpness enhancement processing image signal f2 being obtained from the sharpness enhancement image processing,
iii) calculating a difference |f1xe2x88x92fin| between image signal components of the graininess suppression processing image signal f1 and the original image signal fin, which image signal components represent corresponding pixels in two images represented by the graininess suppression processing image signal f1 and the original image signal fin,
iv) comparing the calculated difference and a predetermined threshold value T with each other,
v) with respect to pixels, which have been found, as a result of the comparison, to be associated with the difference smaller than the predetermined threshold value (|f1xe2x88x92fin| less than T), obtaining the processed image signal fout with the operation processing described above, i.e., with operation processing carried out on image signal components of the graininess suppression processing image signal f1 and the sharpness enhancement processing image signal f2, which image signal components represent corresponding pixels in two images represented by the graininess suppression processing image signal f1 and the sharpness enhancement processing image signal f2, and
vi) with respect to pixels, which have been found, as a result of the comparison, to be associated with the difference larger than the predetermined threshold value (|f1xe2x88x92fin| greater than T), taking the original image signal fin, an enhancement processing signal Finxe2x80x2, which is obtained by carrying out enhancement processing on the original image signal fin, or the sharpness enhancement processing image signal f2 as the processed image signal fout.
In the second image processing method in accordance with the present invention, with respect to pixels, which are associated with the difference |f1xe2x88x92fin| equal to the threshold value T (|f1xe2x88x92fin|=T), the operation processing employed with respect to the pixels, which are associated with the difference smaller than the threshold value T, may be carried out. Alternatively, with respect to such pixels, the processing employed with respect to the pixels, which are associated with the difference larger than the threshold value T, may be carried out. However, in such cases, the former processing (i.e., the operation processing employed with respect to the pixels, which are associated with the difference smaller than the threshold value T) should preferably be employed.
Also, the details of the operation processing may be selected from a plurality of kinds of predetermined operations and in accordance with, for example, the kind of the image or the position of a picture element. Alternatively, information representing the details of the operation processing may be inputted from the exterior each time the processing is to be carried out.
Specifically, in the second image processing method in accordance with the present invention, as the operation processing, the operation processing described above, which is defined with Formula (1) shown below, may be employed:
fout=w1xc2x7f1+w2xc2x7f2xe2x80x83xe2x80x83(1)
wherein w1 and w2 represent weight factors, w1+w2=1 (0 less than w1 less than 1, 0 less than w2 less than 1), fout represents the processed image signal, f1 represents the graininess suppression processing image signal, and f2 represents the sharpness enhancement processing image signal.
Specifically, with the operation processing defined with Formula (1), the processed image signal fout is obtained by weighting the graininess suppression processing image signal f1 and the sharpness enhancement processing image signal f2 and adding the weighted image signals to each other. In the operation processing defined with Formula (1) shown above, the values of the weight factors w1 and w2 may be altered in accordance with the kind of the image, the position of a picture element, the relationship with the value of the original image signal, or the like. Alternatively, the weight factors common to all of the pixels may be set.
Each of the weight factors w1 and w2 should preferably take a value falling within the range of 0.2 to 0.8, should more preferably take a value falling within the range of 0.3 to 0.7, and should most preferably take a value falling within the range of 0.4 to 0.6.
Alternatively, in the second image processing method in accordance with the present invention, as the operation processing, the operation processing described above, which is defined with Formula (2) shown below, may be employed:
fout={|f1xe2x88x92fin|(f1xe2x88x92fin)+|f2xe2x88x92fin|(f2xe2x88x92fin)}/{|f1xe2x88x92fin|+|f2xe2x88x92fin|}+finxe2x80x83xe2x80x83(2)
wherein fin represents the original image signal, fout represents the processed image signal, f1 represents the graininess suppression processing image signal, and f2 represents the sharpness enhancement processing image signal.
Further, as in the first image processing method in accordance with the present invention, as the graininess suppression image processing, it is possible to employ median filter processing, hysteresis smoothing processing, noise removal processing with iteration, graininess suppression processing (smoothing processing) utilizing a morphology operation, or the like. As the sharpness enhancement image processing, it is possible to employ unsharp masking processing, high-band enhancement filter processing, processing disclosed in, for example, U.S. Pat. No. 5,739,922, wherein an image signal is separated into low frequency components, middle frequency components, and high frequency components, wherein enhancement and suppression processing is carried out such that the high frequency components may be enhanced and the middle frequency components may be suppressed, and wherein the high frequency components and the middle frequency components, which have been obtained from the enhancement and suppression processing, and the low frequency components are combined with one another, or the like. As the graininess suppression image processing and the sharpness enhancement image processing, there should preferably be employed a combination of the graininess suppression processing (smoothing processing), which utilizes a morphology operation, and the sharpness enhancement processing (disclosed in, for example, U.S. Pat. No. 5,739,922, wherein the enhancement and suppression processing is carried out such that the high frequency components of an image may be enhanced and the middle frequency components of the image may be suppressed, and wherein the high frequency components and the middle frequency components, which have been obtained from the enhancement and suppression processing, and the low frequency components are combined with one another.
The predetermined threshold value T, which is compared with the difference |f1xe2x88x92fin| between the graininess suppression processing image signal f1 and the original image signal fin, varies in accordance with, for example, the kind of the recording medium, on which the original image is recorded, and details (frequency, and the like) of an image area, which is not to be subjected to graininess suppression, such as a catch lights area, which is reflected in the pupil of the eye of a human body and is considered to be essential to portraits. From the results of experiments carried out by the inventors and experience of the inventors, the threshold value T should preferably be equal to a value, xcex1xc3x97D (where 0.01xe2x89xa6xcex1xe2x89xa60.20), which falls within the range of 1% to 20% of the number D of gradation levels, which the original image signal fin can take. (In cases where the original image signal can take 10-bit gradation levels, D=210=1,024 gradation levels, which correspond to signal values of 0 to 1,023.) The threshold value T should more preferably be equal to a value, xcex1xc3x97D (where 0.02xe2x89xa6xcex1xe2x89xa60.10), which falls within the range of 2% to 10% of the number D of gradation levels, which the original image signal fin can take. In cases where the threshold value T takes a value falling within the range described above, the image area, which is not to be subjected to graininess suppression, such as a catch lights area, can be prevented from being subjected to the graininess suppression processing, and a processed image can be obtained, in which both the graininess suppression and the sharpness enhancement have been achieved more efficiently.
The present invention still further provides a second image processing apparatus for carrying out the second image processing method in accordance with the present invention. Specifically, the present invention still further provides a second image processing apparatus, wherein predetermined processing is carried out on an original image signal, which is made up of a series of image signal components defining values of pixels constituting an original image, and a processed image signal is thereby obtained, the apparatus comprising:
i) a graininess suppression processing means for carrying out graininess suppression image processing on the original image signal, a graininess suppression processing image signal being obtained from the graininess suppression image processing,
ii) a sharpness enhancement processing means for carrying out sharpness enhancement image processing on the original image signal, a sharpness enhancement processing image signal being obtained from the sharpness enhancement image processing,
iii) a comparison means for calculating a difference between image signal components of the graininess suppression processing image signal, which has been obtained from the graininess suppression processing means, and the original image signal, which image signal components represent corresponding pixels in two images represented by the graininess suppression processing image signal and the original image signal fin, and comparing the calculated difference and a predetermined threshold value with each other, and
iv) a signal processing means for carrying out processing, which comprises the steps of:
a) with respect to pixels, which have been found, as a result of the comparison made by the comparison means, to be associated with the difference smaller than the predetermined threshold value, obtaining the processed image signal by carrying out operation processing on image signal components of the graininess suppression processing image signal and the sharpness enhancement processing image signal having been obtained from the sharpness enhancement processing means, which image signal components represent corresponding pixels in two images represented by the graininess suppression processing image signal and the sharpness enhancement processing image signal, and
b) with respect to pixels, which have been found, as a result of the comparison, to be associated with the difference larger than the predetermined threshold value, taking the original image signal, an enhancement processing signal, which is obtained by carrying out enhancement processing on the original image signal, or the sharpness enhancement processing image signal as the processed image signal.
In the second image processing apparatus in accordance with the present invention, as the operation processing, which is carried out by the signal processing means, the operation processing described above, which is defined with Formula (1) or Formula (2) shown below, should preferably be employed.
fout=w1xc2x7f1+w2xc2x7f2xe2x80x83xe2x80x83(1)
wherein w1 and w2 represent weight factors, w1+w2=1 (0 less than w1 less than 1, 0 less than w2 less than 1), fout represents the processed image signal, f1 represents the graininess suppression processing image signal, and f2 represents the sharpness enhancement processing image signal.
fout={|f1xe2x88x92fin|(f1xe2x88x92fin)+|f2xe2x88x92fin|(f2xe2x88x92fin)}/{|f1xe2x88x92fin|+|f2xe2x88x92fin|}+finxe2x80x83xe2x80x83(2)
wherein fin represents the original image signal, fout represents the processed image signal, f1 represents the graininess suppression processing image signal, and f2 represents the sharpness enhancement processing image signal.
The predetermined threshold value T, which is compared with the difference |f1xe2x88x92fin| between the graininess suppression processing image signal f1 and the original image signal fin, should preferably be equal to a value, xcex1xc3x97D (where 0.01xe2x89xa6xcex1xe2x89xa60.20), which falls within the range of 1% to 20% of the number D of gradation levels, which the original image signal fin can take. The threshold value T should more preferably be equal to a value, xcex1xc3x97D (where 0.02xe2x89xa6xcex1xe2x89xa60.10), which falls within the range of 2% to 10% of the number D of gradation levels, which the original image signal fin can take.
Third and fourth image processing methods in accordance with the present invention are characterized by carrying out opening processing in accordance with a morphology operation and closing processing in accordance with a morphology operation, respectively, on an original image signal, carrying out operation processing on image signal components of two image signals having been obtained from the opening processing and the closing processing, which image signal components represent corresponding pixels in two images represented by the two image signals, and thereby obtaining a processed image signal (in this case, a graininess suppression processing image signal). The third and fourth image processing methods in accordance with the present invention are appropriate for the graininess suppression image processing utilizing the morphology operation described above.
Specifically, the present invention also provides a third image processing method, wherein predetermined processing is carried out on an original image signal (or each of a luminance signal and color difference signals, each of original image signals representing an original color image, or the like) fin, which is made up of a series of image signal components defining values of pixels constituting an original image, and a processed image signal (in this case, a graininess suppression processing image signal) fout is thereby obtained, the method comprising the steps of:
i) carrying out opening processing in accordance with a morphology operation on the original image signal fin, an opening processing image signal fopn being obtained from the opening processing,
ii) carrying out closing processing in accordance with the morphology operation on the original image signal fin, a closing processing image signal fcls being obtained from the closing processing, and
iii) carrying out processing for obtaining the processed image signal (in this case, the graininess suppression processing image signal) fout in accordance with the opening processing image signal fopn and the closing processing image signal fcls, the processed image signal fout being obtained with operation processing g(fopn, fcls) (=fout) carried out on image signal components of the opening processing image signal fopn and the closing processing image signal fcls, which image signal components represent corresponding pixels in two images represented by the opening processing image signal fopn and the closing processing image signal fcls, the operation processing being defined with Formula (3):                     fout        =                  {                                                    fopn                                                              (                                                            if                      ⁢                                              xe2x80x83                                            ⁢                      fcls                                        =                    fin                                    )                                                                                    fcls                                                              (                                                            if                      ⁢                                              xe2x80x83                                            ⁢                      fopn                                        =                    fin                                    )                                                                                                                          w1                    ·                    fopn                                    +                                      w2                    ·                    fcls                                                                                                (                  else                  )                                                                                        (        3        )            
wherein w1 and w2 represent weight factors, w1+w2=1 (0 less than w1 less than 1, 0 less than w2 less than 1), fin represents the original image signal, fout represents the processed image signal (in this case, the graininess suppression processing image signal), fopn represents the opening processing image signal, and fcls represents the closing processing image signal.
Specifically, with the operation processing defined with Formula (3), as for a picture element, at which the value of the closing processing image signal fcls is equal to the value of the original image signal fin, the value of the opening processing image signal fopn is taken as the value of the processed image signal fout. As for a picture element, at which the value of the opening processing image signal fopn is equal to the value of the original image signal fin, the value of the closing processing image signal fcls is taken as the value of the processed image signal fout. As for a picture element, at which the value of the closing processing image signal fcls and the value of the opening processing image signal fopn are not equal to the value of the original image signal fin, the addition processing through the weighting with the weight factors w1 and w2 is carried out on the closing processing image signal fcls and the opening processing image signal fopn.
As for how to determine the weight factors w1 and w2, one of various techniques for determination may be employed. Each of the weight factors w1 and w2 should preferably take a value falling within the range of 0.2 to 0.8, and should more preferably take a value falling within the range of 0.3 to 0.7. Each of the weight factors w1 and w2 should even further preferably take a value falling within the range of 0.4 to 0.6, and should most preferably take a value of 0.5.
The present invention further provides a fourth image processing method, wherein predetermined processing is carried out on an original image signal (or each of a luminance signal and color difference signals, each of original image signals representing an original color image, or the like) fin, which is made up of a series of image signal components defining values of pixels constituting an original image, and a processed image signal (in this case, a graininess suppression processing image signal) fout is thereby obtained, the method comprising the steps of:
i) carrying out opening processing in accordance with a morphology operation on the original image signal fin, an opening processing image signal fopn being obtained from the opening processing,
ii) carrying out closing processing in accordance with the morphology operation on the original image signal fin, a closing processing image signal fcls being obtained from the closing processing, and
iii) carrying out processing for obtaining the processed image signal (in this case, the graininess suppression processing image signal) fout in accordance with the opening processing image signal fopn and the closing processing image signal fcls, the processed image signal fout being obtained with operation processing g(fopn, fcls) (=fout) carried out on image signal components of the opening processing image signal fopn and the closing processing image signal fcls, which image signal components represent corresponding pixels in two images represented by the opening processing image signal fopn and the closing processing image signal fcls, the operation processing being defined with Formula (4):
fout={|fopnxe2x88x92fin|(fopnxe2x88x92fin)+|fclsxe2x88x92fin|(fclsxe2x88x92fin)}/{|fopnxe2x88x92fin|+|fclsxe2x88x92fin|}+finxe2x80x83xe2x80x83(4)
Specifically, with the operation processing defined with Formula (4), as for a picture element, at which the value of the opening processing image signal fopn is equal to the value of the original image signal fin, the value of the closing processing image signal fcls is taken as the value of the processed image signal fout. As for a picture element, at which the value of the closing processing image signal fcls is equal to the value of the original image signal fin, the value of the opening processing image signal fopn is taken as the value of the processed image signal fout. As for a picture element, at which the value of the opening processing image signal fopn and the value of the closing processing image signal fcls are not equal to the value of the original image signal fin, the addition through weighting with the difference (|fopnxe2x88x92fin| or |fclsxe2x88x92fin|) between the opening processing image signal fopn or the closing processing image signal fcls, whichever has a larger difference with respect to the original image signal fin, (i.e., fopn or fcls) and the original image signal fin is carried out, and the value obtained from the weighting addition is taken as the value of the processed image signal fout.
The present invention still further provides a third image processing apparatus for carrying out the third image processing method in accordance with the present invention. Specifically, the present invention still further provides a third image processing apparatus, wherein predetermined processing is carried out on an original image signal, which is made up of a series of image signal components defining values of pixels constituting an original image, and a processed image signal (in this case, a graininess suppression processing image signal) is thereby obtained, the apparatus comprising:
i) a morphology operation processing means for carrying out opening processing and closing processing in accordance with a morphology operation, respectively, on the original image signal, an opening processing image signal being obtained from the opening processing, a closing processing image signal being obtained from the closing processing, and
ii) a signal processing means for carrying out processing for obtaining the processed image signal (in this case, the graininess suppression processing image signal) in accordance with the opening processing image signal and the closing processing image signal, which have been obtained from the morphology operation processing means, the processed image signal being obtained with operation processing carried out on image signal components of the opening processing image signal and the closing processing image signal, which image signal components represent corresponding pixels in two images represented by the opening processing image signal and the closing processing image signal, the operation processing being defined with Formula (3) shown above.
In the third image processing apparatus in accordance with the present invention, as in the third image processing method in accordance with the present invention, one of various techniques for determining the weight factors w1 and w2, may be employed. Also, each of the weight factors w1 and w2 should preferably take a value falling within the range described above.
The present invention also provides a fourth image processing apparatus for carrying out the fourth image processing method in accordance with the present invention. Specifically, the present invention also provides a fourth image processing apparatus, wherein predetermined processing is carried out on an original image signal, which is made up of a series of image signal components defining values of pixels constituting an original image, and a processed image signal (in this case, a graininess suppression processing image signal) is thereby obtained, the apparatus comprising:
i) a morphology operation processing means for carrying out opening processing and closing processing in accordance with a morphology operation, respectively, on the original image signal, an opening processing image signal being obtained from the opening processing, a closing processing image signal being obtained from the closing processing, and
ii) a signal processing means for carrying out processing for obtaining the processed image signal (in this case, the graininess suppression processing image signal) in accordance with the opening processing image signal and the closing processing image signal, which have been obtained from the morphology operation processing means, the processed image signal being obtained with operation processing carried out on image signal components of the opening processing image signal and the closing processing image signal, which image signal components represent corresponding pixels in two images represented by the opening processing image signal and the closing processing image signal, the operation processing being defined with Formula (4) shown above.
In the third and fourth image processing methods and the third and fourth image processing apparatuses in accordance with the present invention, in order for the processing to be simplified, the structure element employed in the opening processing and the structure element employed in the closing processing should preferably have the same size and the same shape.
With the first image processing method and the first image processing apparatus in accordance with the present invention, each of the sharpness enhancement image processing and the graininess suppression image processing is carried out on the original image signal, and the operation processing is carried out on the image signal components of the two image signals having been obtained from the sharpness enhancement image processing and the graininess suppression image processing, which image signal components represent corresponding pixels in the two images represented by the two image signals. In this manner, the processed image signal is obtained. Therefore, the graininess suppression and the sharpness enhancement for the image can be simultaneously achieved more efficiently than with the conventional various methods or apparatuses.
With the first image processing method and the first image processing apparatus in accordance with the present invention, wherein the details of the operation processing vary for the pixels, instead of being set to be common to all of the pixels, graininess can be suppressed appropriately and sharpness can be enhanced appropriately in accordance with the details of structure patterns embedded in the image.
With the first image processing method and the first image processing apparatus in accordance with the present invention, wherein the operation processing defined with Formula (1) is employed, the weighted mean value of the graininess suppression processing image signal, in which graininess has been suppressed by the graininess suppression image processing, and the sharpness enhancement processing image signal, in which sharpness has been enhanced by the sharpness enhancement image processing, is calculated. In this manner, sharpness can be enhanced, while graininess is being suppressed.
With the first image processing method and the first image processing apparatus in accordance with the present invention, wherein the operation processing defined with Formula (2) is employed, as for a picture element, at which the value of the graininess suppression processing image signal is equal to the value of the original image signal, the value of the sharpness enhancement processing image signal is taken as the value of the processed image signal. As for a picture element, at which the value of the sharpness enhancement processing image signal is equal to the value of the original image signal, the value of the graininess suppression processing image signal is taken as the value of the processed image signal. As for a picture element, at which the value of the graininess suppression processing image signal and the value of the sharpness enhancement processing image signal are not equal to the value of the original image signal, the addition through weighting with the difference between the graininess suppression processing image signal or the sharpness enhancement processing image signal, whichever has a larger difference with respect to the original image signal, and the original image signal is carried out, and the value obtained from the weighting addition is taken as the value of the processed image signal. Therefore, as for a picture element corresponding to a flat image density area (i.e., an image area having little change in image density or luminance), at which graininess is more perceptible than sharpness is, the value of the graininess suppression processing image signal is taken as the value of the image signal representing the picture element, and graininess can thereby be suppressed. Also, as for a picture element corresponding to an image edge area (i.e., an image area having a sharp change in image density or luminance), at which sharpness is more perceptible than graininess is, the value of the sharpness enhancement processing image signal is taken as the value of the image signal representing the picture element, and sharpness can thereby be suppressed.
As the graininess suppression image processing and the sharpness enhancement image processing, there should preferably be employed a combination of the graininess suppression processing, which utilizes the morphology operation, and the sharpness enhancement processing (disclosed in U.S. Pat. No. 5,739,922, wherein the enhancement and suppression processing is carried out such that the high frequency components of the image may be enhanced and the middle frequency components of the image may be suppressed, and wherein the high frequency components and the middle frequency components, which have been obtained from the enhancement and suppression processing, and the low frequency components are combined with one another. In such cases, the effects described below can be obtained.
Specifically, in particular, in cases where the operation processing defined with Formula (2) is employed, if graininess suppression has not been carried out on a sharpness enhancement area, and if sharpness enhancement is carried out extremely, graininess will also be enhanced, and the difference with respect to the original image signal will become larger than the difference between the graininess suppression processing image signal and the original image signal, and consequently graininess suppression will not be effected. Therefore, in cases where Formula (2) is employed, the sharpness enhancement processing disclosed in U.S. Pat. No. 5,739,922, in which the effects of graininess suppression can be obtained to a certain extent also with the sharpness enhancement processing, should preferably be employed.
Also, in cases where the image edge area has become dull due to the graininess suppression processing, such results affect the results of the sharpness enhancement processing. Therefore, the graininess suppression processing utilizing the morphology operation, with which graininess suppression can be carried out such that the image edge area may not become dull, should preferably be employed.
As described above, as the graininess suppression image processing, graininess suppression processing having the edge retaining effects, such as the graininess suppression processing utilizing the morphology operation, may be employed. As the sharpness enhancement image processing, sharpness enhancement processing having a certain extent of graininess suppression effects, such as the sharpness enhancement processing disclosed in U.S. Pat. No. 5,739,922, may be employed. Particularly, in cases where the operation processing defined with Formula (2) is employed, the graininess suppression processing and the sharpness enhancement processing described above should preferably be employed.
With the second image processing method and the second image processing apparatus in accordance with the present invention, each of the sharpness enhancement image processing and the graininess suppression image processing is carried out on the original image signal, and the operation processing is carried out on the image signal components of the two image signals having been obtained from the sharpness enhancement image processing and the graininess suppression image processing, which image signal components represent corresponding pixels in the two images represented by the two image signals. In this manner, the processed image signal is obtained. Therefore, the graininess suppression and the sharpness enhancement for the image can be simultaneously achieved more efficiently than with the conventional various methods or apparatuses. Also, before the operation processing is carried out, a calculation is made to find the difference between the image signal components of the graininess suppression processing image signal and the original image signal, which image signal components represent corresponding pixels in the two images represented by the graininess suppression processing image signal and the original image signal, and the calculated difference and the predetermined threshold value are compared with each other. With respect to pixels, which have been found, as a result of the comparison, to be associated with the difference larger than the predetermined threshold value, i.e. with respect to the pixels at which the degree of graininess suppression is high, the processing containing the aforesaid graininess suppression processing is not carried out, and the original image signal, an enhancement processing signal, which is obtained by carrying out enhancement processing on the original image signal, or the sharpness enhancement processing image signal is taken as the processed image signal. Therefore, the problems can be prevented from occurring in that, for example, the image area, which is not to be subjected to graininess suppression, such as a catch lights area, which is reflected in the pupil of the eye of a human body and is considered to be essential to portraits, is subjected, by mistake, to graininess suppression with the processing containing the aforesaid graininess suppression processing.
With the second image processing method and the second image processing apparatus in accordance with the present invention, wherein the operation processing defined with Formula (1) is employed, the weighted mean value of the graininess suppression processing image signal, in which graininess has been suppressed by the graininess suppression image processing, and the sharpness enhancement processing image signal, in which sharpness has been enhanced by the sharpness enhancement image processing, is calculated. In this manner, sharpness can be enhanced, while graininess is being suppressed.
With the second image processing method and the second image processing apparatus in accordance with the present invention, wherein the operation processing defined with Formula (2) is employed, the value, which is obtained from the weighting addition of the difference between the graininess suppression processing image signal and the original image signal and the difference between the sharpness enhancement processing image signal and the original image signal through weighting with the respective differences, is taken as the value of the processed image signal. Therefore, with respect to the image signal component representing a picture element corresponding to a flat image density area (i.e., an image area having little change in image density or luminance), at which graininess is more perceptible than sharpness is, the rate of contribution of the graininess suppression processing becomes high. Also, with respect to the image signal component representing a picture element corresponding to an image edge area (i.e., an image area having a sharp change in image density or luminance), at which sharpness is more perceptible than graininess is, the rate of contribution of sharpness enhancement processing becomes high. Accordingly, graininess can be suppressed such that the image edge area may not become dull.
As the graininess suppression image processing and the sharpness enhancement image processing, there should preferably be employed a combination of the graininess suppression processing, which utilizes the morphology operation, and the sharpness enhancement processing (disclosed in U.S. Pat. No. 5,739,922, wherein the enhancement and suppression processing is carried out such that the high frequency components of the image may be enhanced and the middle frequency components of the image may be suppressed, and wherein the high frequency components and the middle frequency components, which have been obtained from the enhancement and suppression processing, and the low frequency components are combined with one another. In such cases, the effects described below can be obtained.
Specifically, in particular, in cases where the operation processing defined with Formula (2) is employed, if graininess suppression has not been carried out on a sharpness enhancement area, and if sharpness enhancement is carried out extremely, graininess will also be enhanced, and the difference with respect to the original image signal will become larger than the difference between the graininess suppression processing image signal and the original image signal, and consequently graininess suppression will not be effected. Therefore, in cases where Formula (2) is employed, the sharpness enhancement processing disclosed in U.S. Pat. No. 5,739,922, in which the effects of graininess suppression can be obtained to a certain extent also with the sharpness enhancement processing, should preferably be employed.
With the third and fourth image processing methods and the third and fourth image processing apparatuses in accordance with the present invention, each of the opening processing and the closing processing in accordance with the morphology operation is carried out on the original image signal. The opening processing image signal, in which a convex image density change area (hereinbelow referred to as the positive noise or positive graininess) has been suppressed, is obtained from the opening processing. Also, the closing processing image signal, in which a concave image density change area (hereinbelow referred to as the negative noise or negative graininess) has been suppressed, is obtained from the closing processing. The operation processing is then carried out on the image signal components of the two image signals having been obtained from the opening processing and the closing processing, which image signal components represent corresponding pixels in the two images represented by the two image signals, and the processed image signal (in this case, the graininess suppression processing image signal) is thereby obtained. Therefore, positive noise and negative noise contained in the original image can be suppressed accurately.
Also, with the third and fourth image processing methods and the third and fourth image processing apparatuses in accordance with the present invention, wherein the details of the operation processing vary for the pixels, instead of being set to be common to all of the pixels, noise can be suppressed appropriately in accordance with the details of structure patterns embedded in the image. Therefore, graininess can be suppressed more accurately, and the degree of visual graininess suppression can be kept higher than with conventional processing technique in which the mean value of the opening processing image signal and the closing processing image signal is employed for every picture element (i.e., the technique described in xe2x80x9cDetail-Preserving Smoothing with Morphology and Fuzzy Reasoning,xe2x80x9d Journal of Electronic Imaging, Vol. 5(3),pp. 396-401, Jul. 1996). Also, graininess can be suppressed with the technique simpler than a conventional technique in which a large number of reference tables are utilized (i.e., the technique referred to as the xe2x80x9cTsukasamoto""s Fuzzy Modelsxe2x80x9d and reported in the literature described above, wherein the difference between a closing processing image signal and an opening processing image signal is compared with a certain table, the corresponding area is judged as being grainy in cases where the difference is larger than a certain threshold value, the corresponding area is judged as being not grainy and being a fine structure pattern in the image in cases where the difference is smaller than the threshold value, and the area having been judged as being grainy is processed).
With the third image processing method and the third image processing apparatus in accordance with the present invention, wherein the operation processing defined with Formula (3) is employed, as for a picture element representing negative noise, which is not suppressed by the opening processing (i.e., the picture element, at which the value of the opening processing image signal is equal to the value of the original image signal), the value of the closing processing image signal is taken as the value of the processed image signal. Therefore, as for such a picture element, the processed image signal is obtained wherein negative noise has been suppressed. As for a picture element representing positive noise, which is not suppressed by the closing processing (i.e., the picture element, at which the value of the closing processing image signal is equal to the value of the original image signal), the value of the opening processing image signal is taken as the value of the processed image signal. Therefore, as for such a picture element, the processed image signal is obtained wherein positive noise has been suppressed. Accordingly, both the positive noise and the negative noise can be suppressed. As for a picture element, at which the value of the opening processing image signal and the value of the closing processing image signal are not equal to the value of the original image signal, the weighted mean value of the opening processing image signal and the closing processing image signal is taken as the value of the processed image signal. In this manner, positive noise and negative noise can be suppressed appropriately. In particular, in cases where the weighting ratio in the calculation of the weighted mean value is set to be 5:5, optimum graininess suppression can be achieved.
With the fourth image processing method and the fourth image processing apparatus in accordance with the present invention, wherein the operation processing defined with Formula (4) is employed, as for a picture element representing negative noise, which is not suppressed by the opening processing (i.e., the picture element, at which the value of the opening processing image signal is equal to the value of the original image signal), the value of the closing processing image signal is taken as the value of the processed image signal. Therefore, as for such a picture element, the processed image signal is obtained wherein negative noise has been suppressed. As for a picture element representing positive noise, which is not suppressed by the closing processing (i.e., the picture element, at which the value of the closing processing image signal is equal to the value of the original image signal), the value of the opening processing image signal is taken as the value of the processed image signal. Therefore, as for such a picture element, the processed image signal is obtained wherein positive noise has been suppressed. Accordingly, both the positive noise and the negative noise can be suppressed. As for a picture element, at which the value of the opening processing image signal and the value of the closing processing image signal are not equal to the value of the original image signal, the addition through weighting with the difference between the opening processing image signal fopn or the closing processing image signal fcls, whichever has a larger difference with respect to the original image signal fin, and the original image signal fin is carried out, and the value obtained from the weighting addition is taken as the value of the processed image signal. In this manner, positive noise and negative noise can be suppressed appropriately.